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Executive Summary 


Over the next three years, the California State Board of Education (SBE) and the California 
Department of Education (CDE) will refine the college and career readiness (CCR) measures in 
the state’s school accountability system. Determining which measures to use, given that they 
need to be robust, useful, equitable, and cost effective—and effectively measure a complex set 
of factors—has proven to be a challenging process for these state agencies to date. 


To inform their work, this report captures how state and county officials, high school teachers, 
community college instructors, CCR program administrators, high school students, and 
community college students view California's current CCR measures. It focuses particularly on 
what teachers and students think makes certain CCR programs successful, based on their 
experiences. Our study answers two questions: How do officials from the state and county 
offices of education view CCR programs and the state’s College/Career Indicator (CCI)? What 
are students’, teachers’, and administrators ’ perspectives on CCR programs? While this 
qualitative study reflects only the views of a small number of stakeholders, their ideas and 
concerns warrant consideration by policymakers. 


Findings: State Officials and County Offices of Education 


Interviewees expressed a desire for more state guidance about CCR. 


Both state and county office of education (COE) interviewees noted the challenge of trying to 
understand what constitutes CCR in the absence of a statewide definition. As many 
interviewees noted, school districts and COEs are creating their own definitions and designing 
their own CCR programs. Most interviewees expressed support for additional, though modest, 
state guidance and an interest in obtaining examples of effective CCR programs. In addition, 
interviewees urged the state to expand the kinds of readiness programs included in the 
California Longitudinal Pupil Achievement Data System (CALPADS) for use in determining 
students’ CCR program completion. 


The term “college and career readiness” often bifurcates the concept of readiness. 


Across the board, interviewees indicated that all students need the knowledge, skills, and 
dispositions to succeed in the contemporary workplace. Several interviewees, however, 
expressed concern that the term “college and career readiness” separates college preparation 
from career preparation. Rather than describing an integrated whole, the term often implies two 
separate entities. 
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Interviewees want more frequent and improved collaboration between K-12 and higher 
education. 


Nearly all interviewees agreed that collaboration between K-12 and postsecondary education is 
essential to implementing effective CCR programs. Several COE interviewees noted that COEs 
often serve as the conveners of regional school districts, institutions of higher education, and 
representatives of the business community. These efforts, they said, rely on the leadership and 
initiative of local individuals rather than on statewide policies, practices, or incentives. This 
finding raises a concern about the ability of COEs in all 58 counties playing this role effectively. 


Measuring CCR effectively and constructively is complex. 


Measuring the knowledge, skills, and dispositions needed for success in the workplace is 
complex and controversial. It is easier to devise large-scale proxies for academic readiness; 
doing so for some facets of applied learning and other kinds of knowledge and skills, such as 
communication and teamwork, is complicated. In talking about the development or refinement of 
a state CCl, interviewees across the board were keenly aware of these challenges. While some 
interviewees thought that existing assessments, such as ACT’s WorkKeys, might be able to 
determine workplace skills, they also pointed out that it is not clear what should be measured, 
and how and when. Others voiced concern about adding more standardized tests into the mix, 
both for students and for the educators who would administer them. 


Findings: Students, Teachers, and Administrators 


Applied learning opportunities are essential for effective CCR programs. 


The students, high school teachers, community college faculty, and administrators we 
interviewed emphasized the importance of providing ample opportunities for a wide range of 
students to participate in applied, or work-based, learning opportunities in high school and 
college. These interviewees reported insufficient availability of these opportunities. Nearly all the 
students we interviewed who participated in internships described the value they received in 
being exposed to the world of work. Students we interviewed who did not participate ina CCR 
program regretted they had not done so, and explained that either they were not aware of CCR 
opportunities, or their high schools did not offer such programs. 


Students and teachers expressed deep concern that their high schools did not 
adequately prepare students to succeed in college. 


Many students we interviewed reported that they were inadequately prepared to succeed in 
postsecondary programs. They were surprised by the demands of college, including the 
challenges of studying independently and the rigors of the coursework. Teacher interviewees 
largely echoed the students’ perspectives, particularly noting the difference between preparing 
students for college eligibility, which the teachers said high schools do well, and supporting 
them in high school to do well in college, which has not met with the same success. 
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Measuring CCR skills is challenging. 


When asked about measuring CCR skills, teachers noted that many tools are available to 
assess academic proficiency, but they were skeptical about the availability and effectiveness of 
tools to assess career readiness. They also cautioned the state about trying to measure these 
skills for high-stakes purposes. 


Relationships are key. 


Students’ experiences at school often are shaped by the presence of a caring adult who helps 
them feel safe, believes in them, and shows a commitment to their success. Therefore, we were 
not surprised to find that students thought this type of relationship was key to their educational 
journeys. Students told us they faced significant challenges related to poverty, family, and their 
need to work while attending school. Support from a concerned high school faculty or staff 
member working within a CCR program or elsewhere in the school was critical to these 
students. 


Policy Implications 


Considering the findings above, the state, as it works to refine the CCI, should: 


1. Determine who has access to high-quality CCR programs and then broaden that 
access. To ensure access to CCR programs for traditionally underserved students in 
varying parts of the state, the SBE should: (1) request an analysis to gain a better 
understanding of current access to high-quality CCR programs, and (2) support the 
expansion of programs for geographically and demographically underserved groups. 


2. Identify and disseminate information about exemplary CCR programs. The list of 
these model programs ideally would include descriptions of the various elements that 
make them exemplary. School districts and COEs could draw on this information as they 
develop and implement their own CCR programs. 


3. Expand the kinds of CCR programs included in CALPADS. Both locally developed 
programs and national programs need to be included in the California Longitudinal Pupil 
Achievement Data System (CALPADS) or in other reporting mechanisms captured by 
the state’s accountability system. 


4. Incentivize cross-system collaboration. Greater cross-sector collaboration between 
K-12 and higher education and among the education and business sectors will enhance 
opportunities for regions to develop more coherent CCR infrastructures. This type of 
collaboration is evident in pathway programs, but it must be integrated or aligned across 
systems more effectively. Investments in cross-system collaboration, particularly in 
helping COEs to play a convening role, are critical to the sustained development of CCR 
supports for students. 


5. Refine the CCI, but proceed with caution in several areas. First, our interviewees 
expressed serious concerns about the burden of additional testing on an already test- 
heavy system. Second, our interviewees questioned the accuracy of tests that measure 
many of the critical skills, and the knowledge and dispositions, that are needed to 
prepare students for college and careers. Finally, researchers and advocates have 
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repeatedly expressed concerns about the fairness of using CCR measures for anything 
with high stakes attached. 


6. Strengthen the focus of school climate in the CCI. Students in our study were nearly 
unanimous in asserting the centrality of having a caring adult guiding their educational 
careers and their CCR. Relationships are a clear, enabling factor in students’ readiness 
and should be part of any state indicator. Currently, measuring school climate is a 
separate component of the accountability system and one of the multiple measures of 
school effectiveness. Our research suggests that a school climate measure focused on 
student perceptions of the existence of a faculty or staff member concerned about them 
at school should be central to the CCI. 
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Introduction 


By adopting the Common Core State Standards in 2010, the California State Board of 
Education (SBE) signaled to educators that the state’s K-12 system would support greater 
levels of college and career readiness (CCR).* For example, the 11th grade Smarter Balanced 
assessment signals to high school students if they are ready for college-level courses in 
California’s public colleges and universities. But what it means in practice to support greater 
levels of readiness—for students, teachers, and schools—has taken some time to evolve, 
particularly in the realm of career readiness. For example, the SBE is working to determine how 
best to measure readiness as part of the state’s new accountability system.” But it faces 
challenges in developing and refining a College/Career Indicator (CCl) that is practical, 
affordable, and useful to track students over time while also being robust in capturing the 
breadth of knowledge, skills, and dispositions associated with readiness; equitable in accounting 
for the range of effective CCR programs in school districts statewide; and comparable across 
districts. 


Over the past decade, California has supported many initiatives designed to improve CCR in 
high schools, including those focused on providing students with rigorous college preparatory 
courses and courses that integrate academics and career-related knowledge and skills. As a 
result of these kinds of investments, school districts have been implementing a range of 
strategies to support readiness, including various kinds of career pathway programs, 
internships, and work-based learning initiatives.* In March 2016, the SBE adopted an initial CCI. 
In September 2017, the Board adopted a revised CCI and plans to continue to refine this 
measure over time. To the extent that what gets measured tends to affect program 
development, the refinement and use of the state indicator could have important effects on the 
distribution and strength of CCR programs statewide, and on equitable opportunities for CCR 
across California. 


The state’s implementation plan for the CCI over the next three years includes consideration for 
adding additional measures to the indicator. The primary purpose of this report is to inform the 
state’s refinement of the indicator by sharing the perspectives of state and regional officials, K- 
12 and community college teachers, CCR program administrators, and students. 


This report is based on a research study that sought to answer two key questions: 


1. How do officials from the state and county offices of education view CCR programs and 
the CCI? 
2. What are students’, teachers’, and administrators’ perspectives on CCR programs? 
To answer these questions, we first interviewed state and county office of education (COE) 


officials and then conducted interviews and focus groups with high school and community 
college students, teachers, and administrators in six programs throughout California. 
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The following two sections of this report provide brief descriptions of relevant CCR research 
(page 8) and of the California policy context (page 9). Starting on page 12, we present the study’s 
findings from interviews with state and COE officials, followed by the findings from interviews and 
focus groups with students, teachers, and CCR program administrators (page 15). The final 
section offers the policy implications that emerge from this research (page 21). 


Relevant CCR Research 


The literature on CCR is vast,° so we narrowed our scope to three key issues of particular 
relevance to this study. First, there is widespread agreement among researchers that the 
knowledge, skills, and dispositions associated with college readiness and career readiness are 
interconnected, though not identical, and that readiness is best understood and applied through 
integrated approaches to teaching and learning (See What Is College and Career Readiness? 
sidebar). These researchers warn against conceptualizing CCR as two entirely separate sets of 
knowledge, skills, and dispositions.® A major issue that educators grapple with is ensuring that 
students are not tracked by demographic factors into either academic or applied courses. They 
also face challenges in expanding students’ access to high-quality applied options. 


Second, several studies identified key elements of effective CCR pathway programs and 
documented student experiences associated with such programs. For example, SRI’s seven- 
year examination of the California Linked Learning District Initiative identified several benefits of 
this CCR pathway program.’ Other studies also documented the opportunities, challenges, and 
other experiences that students in CCR programs have faced.® Third, many researchers who 
explore readiness measurement have emphasized the significant challenges of this endeavor, 
while others have suggested indicators to assess readiness among high school students. ° 


What Is College and Career Readiness? 


There appears to be growing consensus about what students who are deemed college 
and career ready should know and be able to do, but less clarity about the best ways to 
demonstrate proficiency. The concepts that undergird readiness encompass core 
academic subject matter as well as other factors, such as cooperation, communication, 
critical thinking, synthesizing and integrating information, problem solving, and managing 
time well.*° Those seeking training and education beyond high school also must learn 
how to navigate and pay for their postsecondary experiences. In this report, we refer to 
those skills beyond core academic subjects as “knowledge, skills, and dispositions.” 


We also refer in this report to CCR “programs.” By this we mean programs that integrate 
the knowledge, skills, and dispositions described above, with the purpose of supporting 
student readiness for some form of postsecondary education or training. The programs 
might range from sets of courses to complete curricular pathways. The term “CCR 
program” is not intended to capture a precise way of teaching or learning or a particular 
kind of program. Rather, it is intended to convey the range of courses of study that might 
prepare students to pursue postsecondary education and training. Examples of CCR 
programs are provided in “The California Policy Context” section below. 
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The California Policy Context 


California has enacted a set of policy initiatives for K-12 education that provide 

important context for understanding CCR efforts in the state. These reforms include the 
Common Core State Standards, the Smarter Balanced assessments, the state’s new California 
Career Technical Education Model Curriculum Standards, the Local Control Funding Formula 
(LCFF), the Every Student Succeeds Act (ESSA) state plan, and new state school 
accountability measures. Over the past eight years, the SBE has been focused and deliberate in 
adopting and aligning this series of major reforms for K-12 education.** 


California adopted the Common Core State Standards in 2010. These standards identify the 
knowledge and skills that students should achieve in order to graduate from high school and be 
ready to succeed in postsecondary education and training. In 2013, the state adopted a new 
statewide assessment system, the Smarter Balanced assessments, aligned to the new state 
standards. Since then, many campuses of the California Community Colleges and the California 
State University (CSU) have accepted a score at or above Level 3 on the 11th grade 

Smarter Balanced assessments in math and English language arts/literacy as conditional 
evidence that a student is ready for college-level coursework. 2 


The SBE also adopted newly revised state Career Technical Education (CTE) Model Curriculum 
Standards in 2013. These detailed standards organized by industry sector are accompanied by 
the Standards for Career Ready Practice, a more general list of 12 standards designed to 
identify the fundamental knowledge and skills that students need for postsecondary education 
and training, as well as for the workforce. These standards could be helpful for both academic 
and CTE teachers, but they have not been well publicized for such purposes, and training on 
the standards has been limited primarily to CTE educators. 


The LCFF, enacted in 2013, substantially altered education finance and governance in the 
state. The LCFF shifts many key decisions about allocating education dollars from the state to 
local school districts, requires that local stakeholders (including parents and educators) be 
involved in these decisions, and aims to narrow the achievement gap by providing additional 
funding for historically underrepresented student groups (including low-income students, English 
learners, and foster youth). The LCFF also removes most categorical programs and provides 
districts with greater control over their budgets, including decisions about CCR programming. 


Part of the state’s efforts to enhance local decision-making includes a new state accountability 
system. District-level performance results now are part of the state’s Accountability and 
Continuous Improvement System and reported as multiple measures through the new California 
School Dashboard. The Dashboard, implemented in fall 2017, includes seven state indicators 
(chronic absenteeism, suspension rates, English Learner progress, graduation rate, college and 
career, English language arts, and mathematics), as well as four local indicators (basic 
condition, implementation of standards, parent engagement, and local climate). The 
performance results on each of the state measures are comparable across school districts, 
reflect each district’s current status, and will reflect change on the measure over time. 
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California’s CCR Efforts 


High school students seeking to be eligible to attend the CSU or the UC must complete a series 
of courses called a-g course requirements. In addition, the state supports a variety of other 
ways to help prepare students for postsecondary education and training, including the Early 
Assessment Program, Early Start, and state-level concurrent/dual enrollment. More recently, the 
state has also provided direct support to programs that offer explicit CCR readiness experiences 
for students. For example, California has provided significant state support for the following 
CCR programs over the past decade: 


e California Career Pathways Trust (CCPT) Grants. Competitive grants totaling $500 
million, beginning in 2013, for regional consortia to support career-oriented pathways 
leading to postsecondary education, primarily two-year degrees and certificates. 


e Career Technical Education Incentive (CTE) Grants. Competitive grants totaling $900 
million over three years for education, economic, and workforce development initiatives 
to create new and to sustain existing CTE programs to assist students’ transition to 
employment and postsecondary education. 


e CTE Model Curriculum Standards. Revised and adopted in 2013, standards designed 
to help schools and districts develop high-quality curriculum and instruction so that 
students are college and career ready. 


e California Partnership Academies. Small high school learning communities that 
combine academic and technical education with a career focus. This initiative has 
received state funding since the late 1980s. 


e Linked Learning pilots. Twenty pilot sites throughout California organized as district- 
led initiatives that integrate career-themed pathways with high schools’ organizational 
and instructional practices. 


e Expansion of approved a-g courses. Work by the University of California Office of the 
President (UCOP) and funded by the California Department of Education (CDE) to 
expand the list of approved a-g courses so that it includes more CTE offerings that 
integrate academic and career content. 


These and other programs, designed to help improve CCR, represent an evolution in the state’s 
approach to funding and supporting CCR. 


The CCI Remains a Work in Progress. 
As previously noted, the SBE adopted a revised CCI in September 2017. That revised version 


includes the following measures: 
e results from the Smarter Balanced Summative Assessments; 
e results from the Advanced Placement (AP) exams; 
e results from the International Baccalaureate (IB) exams; 


e completion of the a-g course requirements for admission to the UC and the CSU; 
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e completion of a CTE pathway; and 


e completion of dual enrollment courses. 


The state also identified a three-year implementation plan that includes consideration of 
additional career measures for inclusion in the CCI, such as work-based learning, internships, 
and industry certifications. *° 


A key challenge that the SBE faces as it continues to refine the CCI is capturing the breadth of 
knowledge, skills, and dispositions associated with CCR.“ There are many pathway programs 
in use by schools, but the indicator includes only CTE pathways, primarily because they are 
included in the California Longitudinal Pupil Achievement Data System (CALPADS). A 
stakeholder group called Promoting Authentic College, Career, and Civic Readiness 
Assessment Systems (PACCCRAS) has recommended that the SBE expand the collection of 
data in CALPADS beyond CTE pathways to also include state-funded California Partnership 
Academies, Linked Learning pathways, NAF (formerly National Academy Foundation) 
academies, and district-developed college and career pathway programs.** The SBE may 
consider these or other recommendations as it continues to develop the CCI,*® but currently 
there are many career pathway programs in use by school districts for which student 
performance data will not be captured by the indicator. 


Study Methodology 


This study was divided into two phases. In Phase 1 (fall/winter 2016) we conducted semi- 
structured interviews with 27 state and COE officials familiar with CCR programs and 
initiatives. Interviewees represented members of the CDE, SBE, Office of the Governor, 
Senate Education Committee, California Community College Chancellor's Office, the CSU, 
and 13 COEs. The findings from this phase directly informed the development of research 
questions and methods for Phase 2. 


Based on recommendations from these Phase 1 interviewees, the research team identified 
promising college and career partnerships and programs in different geographic regions of 
the state. Team members secured site visits with six programs in spring 2017. The sites 
chosen were spread across the state and reflected a range of CCR initiatives situated 
within various organizational structures, ranging from school-based programs to a regional 
consortium of districts, community colleges, and industry partners. These programs were 
deemed CCR programs based on their explicit focus on increasing students’ readiness 
skills and their attempts to provide career exploration activities. We did not seek to assess 
program effectiveness; rather, we used these case studies to better understand the 
landscape of CCR activities in California. 


Two-member site visit teams went to each selected site and conducted interviews with a 
total of 34 college faculty members, high school teachers, and related administrators and 
staff. We also conducted 11 student focus groups with 71 primarily community college 
students (a few focus group participants were high school students). In addition, we 
administered surveys to students to gather more information. Each site team then prepared 
debriefs that served as site case studies. For additional information, see Appendix. 
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How State and County Office of Education 
Officials View CCR in California 


The findings in this section are based on interviews with more than two dozen officials from the 
state and county offices of education (COEs). In the interviews, interviewees generally 
coalesced around several important themes and concerns, including the need for a coherent 
statewide approach to CCR. The interviewees also voiced concern about insufficient guidance 
from the state on readiness; the need to integrate college readiness and career readiness, both 
in program implementation and in state policy; the many challenges related to coordinating 
across K-12 and higher education; and the difficulties inherent in measuring readiness 
effectively. 


Interviewees expressed a desire for more state guidance about CCR. 
State and COE officials expressed a range of 


attitudes, and some misgivings, about the The state needs a broad set of 
implications of California’s lack of a statewide parameters that doesn’t rise to the level 
definition of CCR. An issue that recurred with of a definition, but helps districts and 
both sets of interviewees was the difficulty of COEs think about what an effective 


understanding what the state means by CCR program looks like. 
readiness when there is no definition. As 
many interviewees noted, this state of affairs 
leaves school districts and COEs on their own 
to create local definitions and to implement CCR programs that they believe have potential to be 
effective. While some COE officials reported that they appreciated having those responsibilities 
and regard this as an opportunity, others wondered aloud about the inevitable variation in 
program quality across districts and the state, and many state officials expressed this concern 
as well. On balance, most state and COE officials expressed support for additional, though 
modest, state guidance and direction. 


—State official 


Nearly all interviewees acknowledged a tension between the state promulgating a definition of 
readiness to which all districts would adhere and the principle of local control embedded in the 
LCFF. As one COE official told us, “The state is protective of local options because of LCFF.” 
Another interviewee said the state’s CTE standards could serve as a reasonable proxy for a 
definition. However, the interviewee also said these standards are not well known or used by 
educators. 


A few interviewees advocated specifically for a statewide definition of CCR and expressed 
concern that, unless California adopts one, district and county program variation may swamp 
the message the state is trying to send about the importance of readiness to California’s overall 
continuous improvement agenda. These interviewees found it problematic that, as one of them 
said, “The state doesn’t speak with one voice” about CCR. 
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Other interviewees, though not necessarily longing for a statewide definition, said they were left 
somewhat at sea without more direct and deliberate guidance from the state. “We don’t know 
what the state expects” was the theme these interviewees sounded. When probed about the 
kind of state guidance that would be useful, they indicated they were not looking for a specific 
scope and sequence for readiness, but rather for clarity from the state on the elements of 
effective CCR programs. The COE officials we interviewed, for example, said they and the 
districts they serve would benefit from access to examples of CCR programs the state considers 
exemplary. Such examples could help districts begin to chart their own paths to implementing 
CCR programs that are both effective and adapted to their own circumstances and conditions. 


An additional issue arose in our discussions with state and COE leaders. As they described 
programs they believed to be successful in imparting readiness skills to students, what became 
apparent was that many perceive the state’s gathering of student data about career pathways to 
be too narrowly focused on CTE pathways. They said that gathering data about CTE pathways 
is important, but that the state should expand these efforts to include pathway programs such as 
California Partnership Academies, Linked Learning and NAF pathways, as well as district- 
developed college and career pathway programs. 


The term ”college and career readiness” often bifurcates the concept of 
readiness. 


Across the board, interviewees indicated 
that all students need the knowledge, skills, It still feels like a two-tiered system. 
and dispositions generally associated with 
success in a contemporary workplace, 
including oral and written communication 
skills, the ability to cooperate with others, critical thinking skills, problem-solving skills, and 
effective time management. As one state official told us, “We need to prepare students for the 
day after [high school] graduation.” 


—State official 


Several interviewees, however, expressed concern that even the term “college and career 
readiness” separates college preparation from career preparation. Rather than describing an 
integrated whole, the term suggests two separate entities—and this bifurcation tends to be 
replicated in practice, with the unintentional consequence of delineating separate tracks for 
different students. Said one COE official, “When we use a phrase like ‘college and career,’ we 
are separating.” Another noted that the term “misses the overlap between college and career.” A 
third interviewee addressed the implications of this in practice: “There is a cultural shift required. 
[We need to] stop seeing [career readiness] as the stepchild and start prioritizing it in schools 
and integrate it with academics.” 


Several interviewees said that a key aspect of effective CCR implementation involves increasing 
the range of postsecondary opportunities for all students, rather than providing select 
opportunities for different groups of them. As one COE official put it, “I’ve always thought that 
students should have opportunities for everything all the time. | think there should be a lot of on- 
ramps and off-ramps.” Other interviewees similarly emphasized the need for high schools to 
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offer a range of opportunities for students, so that no matter what they choose, no 
postsecondary option would be off limits. 


We also heard that the state’s emphasis on students completing the a-g course requirements 
required for admission to the UC and the CSU does not leave much room in students’ 
schedules for CTE courses that support CCR. Said one COE official, “We want students to have 
options, but the way a-g is structured does not guarantee skill development.” As a state official 
noted, “[In this state] we privilege a-g, [but] career readiness is often an afterthought.” A COE 
official said, “This state has a college mindset.” 


Several interviewees noted that maximizing students’ options and integrating academics and 
career readiness skills will require new kinds of teacher training and different approaches to 
professional development. Major challenges may involve aligning the training and orientation 
that teachers of academic subjects and of CTE receive. As one state official said, “Teachers of 
academic and technical courses work in silos.” 


Interviewees want more collaboration between K-12 and higher education. 
Nearly all interviewees agreed that 


collaboration between K-12 and We design a system that requires 
postsecondary education is essential to students to cross institutional boundaries 
implementing effective CCR programs, and to complete education, and yet the 
almost every interviewee acknowledged that expectations about how instruction is 


such collaboration generally is illusive, delivered across are different. 
except when required for funding purposes. _State official 

Words like “problematic” and “spotty” were 
common descriptors. Many blamed “turf 
issues” between K-12 and higher education. As one COE official explained, “The barriers are 
preconceived ideas, historical conflicts, attitudes from community college about how secondary 
[schools are] operating, and attitudes from secondary [schools] about how postsecondary is 
relating [to K-12].” 


Several COE interviewees noted that their offices often serve as conveners, or as one COE 
official described it, a “convener and prodder.” Some COEs have established regional networks 
that bring together school districts, institutions of higher education, and representatives of the 
local business community, and interviewees said that some of these meetings address CCR. 
These efforts, they said, depend on the leadership and initiative of local individuals rather than 
on statewide policies or practices. In short, where cross-sector CCR collaboration is concerned, 
state and COE officials generally agreed that more would be better—and that this would 
contribute to the success of CCR efforts in the state. 
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Measuring CCR effectively and constructively is complex. 


The means to measure student proficiency 
in academic content exists, though not all College readiness has an agreed upon 
experts agree on the best methods to use. definition...Career readiness [just] has 
Indeed, educators, assessment some skills associated with it. 
professionals, and others continue to debate 
the efficacy of conventional standardized 
tests, performance-based assessments, 
portfolios of student work, and other kinds of measurement tools. The measurement of 
knowledge, skills, and dispositions associated with success in the workplace is even more 
complex and controversial. Appraising students’ ability to communicate effectively, work in 
teams, manage time well, and the like is possible, but difficult, and assessments remain inexact. 


—State official 


Interviewees across the board were keenly aware of these challenges in talking about the 
development of a state CCI. As one state official told us, CCR skills are “much more easily 
observed than assessed.” Another state official emphasized that any measurement indicator 
must accommodate a wide range of CCR approaches throughout the state: “We’re designing a 
multiple-measures approach to the college and career indicator that doesn’t penalize a school 
or over-reward a school for any particular emphasis—acknowledging that there are different 
pathways to college and career, and that it’s not just how well you do on an SAT.” 


Many districts and COEs have been developing their own local CCR programs and their own 
measures for assessing them. A benefit of this approach is that the local measurement tools, 
compared with a statewide measure, may be better aligned with local programming. A drawback 
is that each district must develop its own valid and reliable methods for measuring CCR 
programs. This can be a heavy lift for districts and COEs, and it can result in wide variation in 
program quality and results statewide. 


In addition, some interviewees said that existing assessments, such as ACT’s WorkKeys, can 
effectively measure many workplace skills. But it is not clear what should be measured, and 
how and when. For example, some experts and several of our interviewees have suggested that 
workplace skills be measured after, rather than during, high school. 


Perspectives of Students, Teachers, and 
Administrators About CCR 


To gain a better understanding of CCR, we sought to capture student, teacher, and 
administrator perspectives in the second phase of our research. Students become prepared for 
college and careers in a wide variety of ways. Understanding the challenges they face and the 
supports and experiences that benefit them can inform efforts to improve CCR programs and to 
further develop the CCl. 


Our focus groups and interviews with students, teachers, and program administrators focused 
on student experiences in secondary school and their education and work experiences following 
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high school graduation. We asked students about their opportunities to build CCR skills, the 
challenges and barriers they faced, the skills they believed they needed to succeed in college 
and a career, the keys to developing those skills, and the best ways to measure those skills. 


We asked teachers and program administrators about the academic knowledge and skills and 
the broader knowledge, skills, and dispositions that their students begin with at high school and 
that they develop before graduation. We asked about the curricular and work-based 
opportunities and supports that students receive in high school. We asked teachers to reflect on 
the effects of their individual schools’ classes and programs on CCR. We also elicited their 
suggestions for measures to include in the CCI. 


Applied learning opportunities are essential for effective CCR programs. 


The teachers, administrators, and students 
we interviewed emphasized the importance There are more kids than internships. 
of providing ample opportunities for a wide ... There are just not many companies 
range of students to participate in applied, or in our community. 

work-based, learning opportunities in high 
school and college. Students and faculty 
alike reported that there currently are 
insufficient opportunities. 


—District administrator 


At the CCR programs we visited, the kinds of work-based learning being offered varied from job 
shadowing and other shorter-term experiences to pathways that embedded such learning 
throughout a semester or year. Nearly all students we interviewed described the value they saw 
in being exposed to the world of work. As one student explained: 


“Maybe instead of just taking a course, where you're reading out of a book and 
learning, it'd be so much more beneficial if you had a hands-on experience. So, if 
there is a program where you can go and...participate, even for a week or two, [in] a 
few different types of jobs that you might be interested in, | think that'd be so much 
more beneficial than just a course.” 


Nearly all student interviewees who had participated in a CCR program offered compelling 
stories that demonstrated why the program helped them stay in high school and graduate. For 
example, one student said: 


“Freshman year, | was on the verge of dropping out of high school...! didn't really 
want to do any school or nothing, but when | went to [the program], | found an 
interest in the [welding] shop. And then, by senior year, | was working with animals; | 
found an interest in that. So, it helped me out a lot.” 


Several of the students who did not participate in a CCR program said they wished they'd had 
the opportunity to experience a work-based learning program and, in particular, the extra 
academic supports, smaller learning environment, and closer relationships with teachers and 
peers that such programs can provide. While some schools and districts simply did not offer 
such programs, students sometimes were not aware of the programs that did exist. A few 
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students noted barriers to their participation in a CCR program, including the small window of 
time they had to complete a-g course requirements and enroll in a dual enrollment or pathway 
program. Other students faced challenges with course scheduling. These findings point to the 
need for the state and LEAs to address the barriers students face in enrolling in such programs 
and to expand opportunities for them to participate in high-quality CCR programs. 


The comments by teachers and leaders of CCR programs were consistent with what students 
told us. They recognized the need for in-class instruction and the value of providing them with 
hands-on learning experiences. These experiences included both project-based learning in 
school and work-based opportunities away from school. Several teachers said they had seen 
cases in which student participation in these kinds of experiences during high school made the 
difference between student success and failure. In addition, teachers reported that exposure to 
the world of work had other benefits. A teacher from a health program said: 


“| want students to go on to college or technical school. | also want [to help] students 
to know what they want to do. We have students who end up going to college and 
majoring in something outside of medicine, like music and political science, but the 
academy helped them decide what they didn’t want to do, so it was still beneficial.” 


Some programs we examined offered more extensive hands-on learning opportunities than 
other programs. In general, career academies emphasized exposure to the world of work. A 
teacher in one such program said: 


“Our goal is hands-on learning: mentors, job shadowing, [an] internship junior and 
senior year. Junior year, we focus on sports medicine—anatomy, physiology—...and 
work with the athletic director. Senior year, they can pick an internship in an area of 
interest to them. We have some clinics [where] we'll rotate them through all of the 
departments and some smaller offices, too. They have opportunities to do more 
internships. Some partnerships have summer internships, like [in] dental health.” 


One CCR program that did not emphasize internships still provided multiple opportunities for 
career and college exploration, community service, and coursework designed to build the 
academic and behavioral skills necessary for readiness. This national program emphasizes 
preparation for both college and career, and it views such preparation as equipping students 
with one cohesive set of skills. 
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Students and teachers expressed deep concern that high schools did not 
adequately prepare students for success in college. 


On balance, students in our focus groups 


were positive about their educational Are we trying to make students 
experiences, especially those in CCR college-eligible or college-ready? 
programs. They reported that the teachers in | think we as educators do an okay job 
their CCR programs helped boost their self- getting students college-eligible, but we 
confidence and ability to persevere when do a horrible job getting them college- 


confronting difficult challenges. Still, the ready. 
students were not shy about the challenges 
they faced. Many described inadequate 
preparation and insufficient skills to succeed 
in college or other postsecondary programs. Even among students who were very positive 
about their CCR program, the majority reported that once they entered college, they realized 
they were underprepared. One student said, “They weren't lying when they were telling you it's 
just going to get harder.” 


—High school teacher 


Other students echoed this sentiment and told us they were surprised by the demands of 
college, including the challenges posed by studying more independently (with less classroom 
time than in high school) and the expectations of college professors. As one student said, “I was 
excited [about] the idea of college and knew it would be harder than high school, but didn’t 
expect it to be [So] much more difficult than high school.” The teachers we interviewed generally 
concurred with this view. One teacher said, “When students go to college, what they struggle 
with the most is... the fast pace and rigor, and you don't always get that fast pace and rigor in 
high school.” 


Many students in our focus groups who had not participated in a CCR program in high school 
were quite critical of their experiences in secondary school. While students in pathways 
programs reported that they generally were treated as capable adults (that is, teachers had high 
expectations of them and gave them corresponding responsibilities), some students who had 
not participated in pathways reported that they had not been treated that way, nor asked to 
assume that level of responsibility. 


Likewise, the high school teachers we interviewed said that high school does not adequately 
prepare students for the environment they will face in college. Said one, “We need to do a better 
job not holding [students’] hands so much through the high school experience. We don’t wean 
them [during] junior and senior year like we should. When they get to college, they are slapped 
in the face.” A second teacher pointed to the challenges students face in the transition to 
college, reiterating the faster pace and advanced level of college work: “[In high school], 
students had someone coaching them.” 


Measuring CCR skills is challenging. 


We also found general agreement among students, teachers, and CCR program administrators 
about the knowledge, skills, and dispositions that are crucial for CCR. Overall, interviewees 
pointed to the combination of academic knowledge and related communication and 
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computational skills (e.g., reading, writing, and math); inquiry skills (e.g., questioning techniques 
and deep understanding); organization (e.g., time management and study skills); and 
collaboration (e.g., team work). Some respondents also emphasized other personal qualities 
needed for readiness, such as resiliency, persistence, and self-confidence. Others mentioned 
basic social skills and employability skills, such as dressing appropriately, shaking hands, and 
making eye contact. 


Regardless of a particular program’s focus, high school teachers from each program saw 
common themes among the skills students need to be successful. One teacher commented, 
“Kids need to be able to talk in their field, no matter what industry they go into. They need to be 
conversationally proficient and to be able to communicate....” 


Neither the teachers nor the students in our study were aware of the SBE’s efforts to identify 
CCR measures as part of the new school accountability system. When asked about measuring 
CCR skills, teachers noted that, while many tools are available to assess the academic side of 
readiness, they were skeptical about the availability and effectiveness of tools designed to 
assess Career readiness. They also cautioned the state about trying to measure these skills in 
high stakes ways, given the lack of reliable criteria and data related to career readiness. 


To this latter point, interviewees questioned the state’s ability to develop fair and reasonable 
measures of career readiness. They pointed to the burdensome amount of testing already 
underway and questioned whether it was feasible for the indicator to require yet another test. 
Many teachers acknowledged that their own efforts to measure CCR skills were largely 
subjective, but they said they did not think the state could do better. As one teacher told us, “I 
really don’t know how to measure things like time management skills.” 


The students in our focus groups had not thought much about measuring specific CCR skills, 
and when pressed to identify factors that had helped them succeed, nearly all the students 
emphasized the critical importance of having a caring adult who attended to both their academic 
and social-emotional needs. They also emphasized the importance of having a safe and 
inclusive school climate, positive peer relationships, and committed teachers. In addition, they 
identified specific personal qualities that had helped them succeed, but that they considered 
difficult to measure, such as self-confidence and persistence. 


Relationships are key. 


Often, a key part of students’ experience at 
school is shaped by the presence of a caring 
adult who helps them feel safe and shows a 
commitment to their success. Thus, we were 
not surprised to find that students pointed to 
the importance of relationships in their 
educational lives. Students were frequently 
passionate in their praise for the special 
teachers, counselors, or principals who —Student 
supported them. As one student told us, “I 


| guess what made the program 
successful for the students would 
probably be the teachers, because of 
their dedication for wanting you to 


learn—making sure you got the 
internships you wanted, making sure 
you were still ahead in class. 
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know that | can always go to them... Like, if I'm having problems in my life, | can talk to them 
about it, or if I'm having problems in school, | can talk to them about it.” 


Most students in CCR programs identified a variety of supports they received that they 
considered essential to their progress. At the top of the list were close, supportive relationships 
with their teachers and other adults in their school. One student who had been in the health 
academy said, “They made sure that there were lots of opportunities. | was close to my 
teachers. They pushed...My principal read my scholarship application and helped me sign up 
for college and got me SAT prep.” 


Regardless of whether they were in a CCR program or not, students in our focus groups said 
they faced significant challenges associated with poverty, family problems, and the need to work 
while attending high school. These challenges made support from a caring adult at school even 
more valuable. Nearly all of the students said that their parents never attended college and 
could not help them with the steps needed to go to college. When we asked how many students 
had to work while going to school, almost every hand went up. Moreover, while some students 
said their schools and teachers had their best interests in mind, many reported that conditions at 
their schools impeded their educational progress. These conditions included bullying, the 
shortage of counseling support, too few caring adults, and poor teaching. One student reported 
that one of his teachers told him, “I get paid even if you don’t learn.” 


When asked about the most important component of building students’ CCR skills, teachers 
usually began by talking about specific skills and personal qualities. But when pressed, they, 
too, acknowledged the centrality of relationships. As one teacher told us: 


“The thing they value the most [is] the meaningful adult. That is the beauty of the 
program, and it is family-oriented, and they have a safe person and a safe place with 
a cheerleader who will push them .... You can give the curriculum, but the belief [in 
them] is what gets the students through.” 


This teacher said that measuring the impact of a concerned faculty or staff member on a 
student’s educational life is best done after the student has graduated from high school and had 
some time to reflect. As she told us, there is so much going on in the lives of her students that 
“,.one day they love you, and the next day they hate you.” It is not until students get into 
college or a technical skills program, she said, that they can accurately assess how well- 
prepared they are and how effective their high school teachers were. 


Reflecting on their experiences, teachers and students alike noted the importance, for any 
measure of CCR, of valuing and integrating both college and career preparation opportunities, 
so that more programs will be available to provide students with both academic knowledge and 
exposure to work-based experiences. Interviewees said the opportunities for students to apply 
academic knowledge and skills in real-world settings, to seek out mentors and support services 
in a Supportive school community, and to cultivate those personal qualities that contribute to 
success in both college and career are invaluable for students’ future success. 
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Policy Implications 


Our research is intended to supplement the work of a variety of groups that are helping to inform 
the state’s efforts to refine the CCI and to otherwise support CCR program development and 
students’ acquisition of CCR skills. Collectively, our discussions with students, teachers, and 
state and COE officials revealed important areas of agreement regarding the state’s 
opportunities and challenges, the implications of which suggest actions the state ought to 
consider as it continues to build California’s CCR profile. 


As it works to refine the CCl, the state should: 


1. Determine who has access to high-quality CCR programs and then broaden that 
access. In order to ensure access to CCR programs for traditionally underserved 
students in varying parts of the state, the SBE should: (1) request an analysis to gain a 
better understanding of current access to high-quality CCR programs, and, (2) support 
the expansion of programs for geographically and demographically underserved groups. 
The state needs to understand the geographic spread of the programs and of student 
populations with access to such programs. 


2. Identify and disseminate information about exemplary CCR program models. The 
list of exemplars ideally would include descriptions of the various program elements that 
make them exemplary. Case studies could provide in-depth information about how 
certain elements were developed and implemented and about program successes and 
challenges. Districts and COEs could draw on this information as they develop and 
implement their own CCR programs. 


3. Expand the kinds of CCR programs included in CALPADS. Both locally developed 
programs and national programs need to be included in the California Longitudinal Pupil 
Achievement Data System (CALPADS) or in other reporting mechanisms captured by 
the state’s accountability system. 


4. Incentivize cross-system collaboration. Greater cross-sector collaboration between 
K-12 and higher education and among education and business communities will 
enhance opportunities for regions to develop more coherent and integrated CCR 
infrastructures. This type of collaboration is evident in pathway programs, but it must be 
more effectively integrated or aligned across systems. Cross-system collaboration 
happens most often when there are funding demands or full-time staff whose jobs are 
focused on this cross-sector work. Investments in cross-system collaboration are critical 
to the development of sustained CCR supports for students. 


5. Refine the CCI, but proceed with caution in several areas. First, our interviewees 
expressed serious concerns about the burden of additional testing on an already test- 
heavy system. Second, our interviewees questioned the accuracy of tests and other 
standardized mechanisms to measure many of the critical skills, and the knowledge and 
dispositions students need, to achieve readiness—and the kinds of incentives that 
standardized measures create. Finally, interviewees repeatedly expressed concern 
about the fairness of using CCR measures for high stakes accountability. 
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6. Strengthen the CCl’s focus on school climate. Students in our study were nearly 
unanimous in asserting the centrality of having a caring adult guiding their academic 
careers and their CCR readiness. Relationships are a clear enabling factor in CCR and 
should be part of any indicator. Currently, measuring school climate is a separate 
component of the accountability system and one of the multiple measures of school 
effectiveness. Our research suggests that a school climate measure focused on student 
perceptions of the existence of a caring adult at their school should be central to 
the CCl. 


California has taken important steps in creating overall policy coherence in K-12, in Supporting a 
range of CCR programs, and in beginning to develop ways to measure CCR. As the SBE 
continues to refine the CCI for California’s K-12 accountability system, we hope it will keep front 
and center the perspectives of students, teachers, and others who have participated in or 
managed CCR programs. Along with state and COE officials, these interviewees reported that 
CCR programs are important for students, particularly in offering them applied learning 
opportunities and personal relationships with caring adults. Indeed, our interviewees affirmed 
that CCR is difficult to define and measure, but agreed it would be useful to have some level of 
state guidance in identifying exemplary programs. They also believed equitable access to these 
programs for all students is essential. Finally, state and COE officials said that the state ought to 
expand the kinds of CCR programs about which it gathers student data so as to support the 
development and sustainability of a wide range of effective CCR programming. 


In light of these findings, state officials should proceed with both purpose and caution in 
examining access to CCR programs, disseminating information about effective programs, and 
ensuring that the CCI does not limit the kind of high-quality CCR programs available to students. 
As the SBE moves forward in refining the CCl, this is a pivotal time to strengthen the 
sustainability and reach of exemplary programs - and to expand student access to them across 
California so that more students in every community will become college and career ready. 
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Appendix 


Study Methodology 


Phase One: Interviews with State Officials and County Office of Education 
Administrators (Fall/Winter 2016) 


Research Question 


In the initial stage of research, we conducted exploratory interviews with various state and local 
policy leaders to better understand their views on a number of aspects of CCR. We sought to 
map the policies, initiatives, activities, instructional strategies, and professional development 
intended to improve students’ postsecondary readiness. Specifically, our research addressed 
the following question: 


e How do officials from the state and county offices of education view CCR programs and 
the state’s College/Career Indicator (CCl)? 


Our semi-structured interview protocol included questions about the state’s vision for and 
definition of CCR, the role of the state in promoting and implementing CCR activities, and the 
roles of local K-12 districts, county offices of education, and postsecondary institutions. 


Participants 


In summer 2016, we conducted semi-structured interviews with 27 state and county officials 
familiar with CCR programs and initiatives. The officials were representatives of the CDE, the 
SBE, the Office of the Governor, the Senate Education Committee, the California Community 
College Chancellor’s Office, the CSU, and 13 county offices of education. Interviews lasted 
approximately 40 to 60 minutes and were recorded. 


Data Analysis 


Detailed notes or transcriptions were created from interview recordings, and short summaries of 
key insights were distributed to the research team after each individual interview to inform future 
conversations. Structured debrief guides were then completed for each interview and shared 
with the entire research team in advance of a full-day analysis meeting in which major findings 
and themes were identified and discussed. The findings from this phase of the research directly 
informed the development of research questions and methods for the in-depth case studies 
described in Phase Two. 
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Table 1: Phase one interviewees by role type 


Role Number of 

Interviewees 
California Department of Education 6 
Office of the Governor 1 
State Board of Education aE 
Senate Education Committee consultants 1 
California State University officials 1 
CA Community College Chancellor's Office 1 
COE administrators (representing 13 counties) 16 
Total 27 


Phase Two: Program Interviews and Student Focus Groups (Spring 2017) 


Research Question 


Based on recommendations from state and county officials and other experts in the field, we 
identified promising college and career partnerships and programs in different geographic 
regions of the state as potential case study sites. By conducting in-depth case studies, we 
intended to bring educator and student voices into the policy discussion about the state’s still- 
developing CCI and gain insight about how such an indicator might capture the experiences and 
competencies that students found most valuable in preparing them for postsecondary success. 
The main research question for this phase of the work was: 


e What are students’, teachers’, and administrators’ perspectives on CCR programs? 


Our semi-structured interview protocols for staff members at community colleges, high schools, 
and partnership organizations contained questions about activities that prepare students to 
succeed in college and/or career, organizational capacity to implement college and career 
initiatives, and the sustainability of maintaining innovative programs. Students in focus groups 
were asked retrospective questions about the activities and programs that contributed to CCR 
and perceived barriers to their postsecondary success. 


Sites and Participants 


Based on our initial outreach to eight different college and career partnerships, we secured site 
visits with six programs in spring 2017. These programs were deemed “CCR programs” based 
on their explicit focus on increasing students’ “CCR skills” and on their attempts to provide some 
level of career exploration activity. There was large variation among programs. 


One site utilizes a nationwide CCR program geared to first-generation college students. The 
program focuses on fostering students’ academic and behavioral skills (“CCR skills”) and 
provides students with primarily school-based career exploration opportunities. Another site is 
part of a nationwide network of high schools that integrates internships into the core academic 
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program. Students participate in internships two days a week and are encouraged to 
supplement these experiences with dual enrollment courses and articulated community college 
courses offered on their high school campuses. 


Two of our sites represented direct partnerships between K-12 school districts and local 
community colleges. One of these partnerships centered on a newly created dual enrollment 
program geared toward specific CTE pathways. At the other site, we focused on a sector- 
specific community college pathway program that operated using a cohort model and actively 
recruited from local high schools. Finally, we visited two sites of robust regional partnerships. At 
one site, the county office of education is a hub for a consortium of high school districts and 
community colleges with pathways focused on three sectors aligned with the local economy. 
The consortium’s work, supported by a California Career Pathways Trust grant, aligns 
curriculum between high schools and colleges and includes work-based learning activities. The 
other regional program includes a Linked Learning site that has historically represented a 
partnership between the county office of education and an industry-based organization and 
includes a number of CTE pathway programs. 


Depending on the organizations and institutions involved in an individual program, we 
conducted interviews with county or regional program staff, college faculty and administrators, 
and high school faculty and administrators, for a total of 34 administrator/staff interviews. 
Interview participants were selected via "snowball" sampling, beginning with our contacts in the 
field who recommended faculty at the community colleges, local high schools, and/or regional 
education partnership staff involved in CCR work. 


We also conducted 11 student focus groups attended by a total of 71 primarily community 
college students. The research team asked faculty, administrators, and counselors/advisors to 
help us gain access to/recruit students in pathway programs, resulting in variable participation at 
case study sites. Focus groups lasted approximately one hour, and students were provided 
lunch and $10 gift cards in exchange for their participation. All interviews and focus groups were 
recorded. 
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Table 2: Description of case study sites and data collection 


Student 
Interviews Focus 
Groups 
Site Urbanicity Type ct 
Program County/ a High Number 
sags District | College Senpol - 
egiona 
Staff i nay Faculty | Students 
North Urban HS program — 
integrated 3 2 
internships 
North Suburban District-CC 1 ‘: 1 9 
partnership 
North Urban District-CC 3 5 6 
partnership 
Central Rural Regional CCR 5 5 1 5 23 
Valley initiative 
South Urban County-led CCR ri 5 3 1 91 
initiative 
South Suburban School-based 5 4 10 
CCR program 
5 5 8 16 71 
34 71 


Data Analysis 


After each site visit, the two-person site visit team created transcripts or detailed notes from 
each interview and focus group. Based on analyses of these data, as well as analyses of 
relevant program documentation and websites, each site team produced a detailed case study 
brief. These briefs were shared with the entire research team in advance of a full-day analysis 
meeting, at which we conducted a cross-case analysis to identify common themes and variation 
across the sites. 
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